[SOUND] This lecture is about
learning to rank.
In this lecture, we're going to
continue talking about web search.
In particular, we're going to talk about
using machine running to combine definite
features to improve ranking function.
So the question that we
address in this lecture is
how we can combine many
features to generate a,
a single ranking function
to optimize search results.
In the previous lectures,
we have talked about the,
a number of ways to rank documents.
We have talked about some retrieval
models, like a BM25 or clear light code.
They can generate a content based scores
for matching documents with a query.
And we also talked about
the link-based approaches,
like page rank that can give additional
scores to help us improve ranking.
Now the question now is how can
we combine all these features and
potentially many other
features to do ranking?
And this will be very useful for
ranking web pages not only just to improve
accuracy, but also to improve
the robustness of the ranking function.
So that's it not easy for
a spammer to just perturb a one or
a few features to promote a page.
So the general idea of learning to
rank is to use machine learning to
combine these features to optimize
the weight on different features to
generate the optimal ranking function.
So we would assume that
the given a query document pair,
Q and D,
we can define a number of features.
And these features can vary from
content based features such as
a score of the document it
was respected to the query
according to a retrieval function,
such as BM25 or
Query Light or pivot commands
from a machine or PL2, et cetera.
It can also be linked based
score like PageRank score.
It can be also application of retrieval
models to the anchor text of the page.
Right?
Those are the types of descriptions
of links that pointed to this page.
So these can all be clues about whether
this document is relevant or not.
We can even include a, a feature such
as whether the URL has a [INAUDIBLE],
because this might be the indicator
of home page or entry page.
So, all of these features can then be
combined together to generate the ranking
functions.
The question is of course,
how can we combine them?
In this approach,
we simply hypothesize that the probability
that this document is random to this query
is a function of all these features.
So we can hypothesize this
that the probability of
relevance is related to these
features through a particular
form of the function
that has some parameters.
These parameters can
control the influence of
different features on the final relevance.
This is of course, just a assumption.
Whether this assumption really makes
sense is still a, a big question.
However, you have to empirically
evaluate the, the, the function.
But by hypothesizing that the relevance
is related to those features
in the particular way, we can then
combine these futures to generate
the potentially more powerful ranking
function, a more robust ranking function.
Naturally, the next question is how
do we estimate loose parameters?
You know, how do we know which
features should have high weight and
which features should have low weight?
So this is a task of training or learning.
All right.
So,
in this approach what we will
do is use some training data.
Those are the data that
have been judged by users.
So that we already know
the relevance judgments.
We already know which documents should
be rather high for which queries and
this information can be based on
real judgments by users or can,
this can also be approximated by just
using click through information.
Where we can assume the clicked documents
are better than the skipped documents or
clicked documents are relevant and
the skipped documents are not relevant.
So, in general, the fit such hypothesize
ranging function to the training day,
meaning that we will try to optimize its
retrieval accuracy on the training data.
And we adjust these parameters to see
how we can optimize the performance
of the function on the training data in
terms of some measure such as map or NDCG.
So the training data would
look like a table of tuples.
H-tuple it has three elements, the query,
the document and the judgment.
So, it looks very much like
our relevance judgment that we
talked about in evaluation
of retrieval systems.
[MUSIC]

